Digital image analysis of reflecting markers

ABSTRACT

The subject of the invention is a system for photogrammetric determination of centers of gravity of signal markers in moving objects. For reduction of the herein produced image data prior to the actual image measurement in a computer, patterns of adjacent pixels across lines are evaluated in a data reduction step. The various possibilities of light/dark constellations of groups of adjacent pixels serve therein as input signals of an evaluation logic, which is an end automation. Various patterns trigger transitions between the finite conditions of the automation. Depending upon the current condition of the final automation and actual detected pixel patterns the final automation causes varying operations for identification of characteristic features of the signal marks and for data technological combining of the parameters necessary for determining centers of gravity.

[0001] The present invention concerns image recognition systems foridentifying particular information content from the image signal of anelectronic camera. In particular, the invention concerns thephotogrammetric ascertainment of the center of gravity of the opticalsignal markers in rapidly moving objects.

[0002] Systems of this type are employed for example in the analysis ofmovement patterns of dynamic processes (deformation processing duringcrash tests, detail studies of motor functions of human movement, etc.).For this, optical signal markers with a specified shape and size (forexample circular disks) are applied onto various locations of the objectbeing observed. Therein the surface of this signal marker is typicallydesigned such that it reflects the impinging light in a characteristicmanner.

[0003] Each of the individual images provided by a video camera at shorttime intervals, comprised of a multiplicity of linear sequences ofindividual image points (pixels), must be evaluated with respect to thesignal components which are associated with the signal markers. In suchan image measurement of retro-reflecting circular signal markers twobasic problems must be solved: first, associated fields must be analyzedand characterized and subsequently the center of each area (=center ofgravity the signal mark) must be determined. For both problems a purelysoftware based evaluation process is known:

[0004] Correlation Analysis

[0005] Since retro-reflecting signal markers exhibit a good imagecontrast, the correlation analysis can be carried out using a binaryversion (that is, only black/white pixel values) of the respectiveimage. The most complete and at the same time efficient solution of thisproblem is supplied by the BCC-algorithm (Binary Connected Components)of MANDLER & OBERLANDER (Informatik Fachberichte 254, pages 248-254,Springer 1990). With only one single pass it can determine the edges ofall associated fields and their correlation or interrelationship in animage. This applies also for multiple subdivided areas. A special setupfor photogrammetry was described by MAAS (Digital Photogrammetry inThree-Dimensional Flow Measurement Technology, PhD thesis, ETH Zurich,1992).

[0006] Point Measurement

[0007] If all areas are marked, then a process for measuring of pointscan be applied to them, as described for example by LUHMANN (Near AreaPhotogrammetry, Principles of Method Applications, Wichmann Publishers,2000) or SHORTIS ET AL. (Practical Testing of Precision and Accuracy ofTarget Image Centering Algorithms, SPIE 2598, pages 65-76, 1995).

[0008] In the most simple case one determines the center of gravity(u,v) of the binarized image b as center value of the image coordinatesof all points b(u,v) ≡ 1 in the environment of the mark: $\begin{matrix}{\overset{\_}{u} = {{\frac{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{u \cdot {b\left( {u,v} \right)}}}}{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{b\left( {u,v} \right)}}}\quad \overset{\_}{v}} = \frac{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{v \cdot {b\left( {u,v} \right)}}}}{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{b\left( {u,v} \right)}}}}} & (1)\end{matrix}$

[0009] Another possibility is the calculation of a gray-value weightedcenter of gravity (Weighted Center of Gravity) of the gray image gaccording to the formula: $\begin{matrix}{\overset{\_}{u} = {{\frac{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{u \cdot {g\left( {u,v} \right)} \cdot {b\left( {u,b} \right)}}}}{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{{g\left( {u,v} \right)} \cdot {b\left( {u,b} \right)}}}}\quad \overset{\_}{v}} = \frac{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{v \cdot {g\left( {u,v} \right)} \cdot {b\left( {u,b} \right)}}}}{\sum\limits_{u = u_{\min}}^{u_{\max}}{\sum\limits_{v = v_{\min}}^{v_{\max}}{{g\left( {u,v} \right)} \cdot {b\left( {u,b} \right)}}}}}} & (2)\end{matrix}$

[0010] The multiplication with the binarized image b serves forselection of the image point, which is associated with the mark.

[0011] More complex is the so-called Template Matching. Therein thecross-correlation k between a predetermined pattern m(u,v) and the imageg(u,v) is calculated. $\begin{matrix}{{k\left( {u^{\prime},v^{\prime}} \right)} = {\int_{u_{\min}}^{u_{\max}}{\int_{v_{\min}}^{v_{\max}}{{{m\left( {{u - u_{\min}},{v - v_{\min}}} \right)} \cdot {g\left( {{u - u^{\prime}},{v - v^{\prime}}} \right)}}{v}\quad {u}}}}} & (3)\end{matrix}$

[0012] The maximum of the correlation function k provides the locationof the mark center. Since the marks are usually distorted in perspectiveand appear to have various sizes in the image, it is not sufficientmerely to compare them with a pattern or template. One must calculatethe correlation with a complete palette of various models and find theresult of the “best fit”. Thereby the calculation process requiresrelatively high calculation time. As models, conventionally employed areellipses or rectangles of various size with various edge sharpness.

[0013] Image evaluation processes for identification of the centers ofgravity of signal marks using exclusively software solutions are inprinciple relatively slow, since in the case of high resolution images avery high rate stream of data occurs in rapid sequence, of which theevaluation is appropriately time consuming. An HDTV camera with aresolution 1280×1024 image points and an image rate of 25 images persecond provides, for example, an average data rate of 32 million pointsper second, with a typical transmission rate of 40 million points persecond. With the present limitation of PCI-busses for standard computersof around 100 MByte per second this presents a substantial load ordemand on the computer architectural even considering only purely thedata transmission. In a simultaneous processing—or further, in the caseof connection of multiple cameras—a conventional microprocessor with aconventional operating system does not even come into consideration.

[0014] In order to estimate the maximal load of a computer, onecalculates the number n_(t) of the clock cycles, available forprocessing of each image point: $\begin{matrix}{n_{t} = \frac{f}{r_{k}}} & (4)\end{matrix}$

[0015] with the clock frequency f of the microprocessor and thetransmission rate r_(k) of the camera.

[0016] If one uses for r_(k) the above proposed transmission rate of 40million image points per second, that is, 40 MHz, and as f 1000 MHz,then there is available for each image point essentially n_(t)=25 clockcycles. This is more rapid for an employment in this area, however isnot sufficient for carrying out high-resolution image evaluation. Such avalue for n_(t) should be significantly greater than 1000, in particularwhen one considers that not all clock cycles are available for processoroperation, but rather a part is lost while waiting for hard driveaccess.

[0017] So that the system does not operate at its limits, there shouldin addition be factored in a generous reserve of α=50% of the cycles forthe simultaneously occurring tasks of the operating system. For the caseof a simultaneous operation of c camera the number of clock cyclesavailable per image point is reduced even further: $\begin{matrix}{{n_{t}\left( {1 - \alpha} \right)} \cdot \frac{f}{c \cdot r_{k}}} & (5)\end{matrix}$

[0018] With this estimation it can be seen, that already prior to aprocessing in a computer already a drastic reduction of the signalsupplied by the camera must occur.

[0019] If one solves the equation (5) according to r_(k), then one cancalculate therewith the maximal data rate r_(max), which may besubmitted for evaluation in the computer: $\begin{matrix}{r_{\max} = \frac{f \cdot \left( {1 - \alpha} \right)}{c \cdot n_{t}}} & (6)\end{matrix}$

[0020] Of interest herein is the data reduction factor γ, which resultsfrom the relationship of r_(k) to r_(max): $\begin{matrix}{\gamma = {\frac{r_{k}}{r_{\max}} = \frac{r_{k} \cdot c \cdot n_{t}}{f \cdot \left( {1 - \alpha} \right)}}} & (7)\end{matrix}$

[0021] In a system with c=4 cameras, α0.5, f=1000 MHz, r_(k)=40 MHz andn_(t)≧1000 cycles per image point there results a data reduction factorof:

γ≧320.   (8)

[0022] This means that, for example, the information content of an imagewith originally one MByte storage requirement must be represented with amaximum of 3200 byte, before the computer can switch to furtherevaluation.

[0023] The searched information, that is, the coordinates of the marksto be measured, are strongly redundantly coded in the original grayimage. In order to achieve the desired data reduction, various methodsare employed:

[0024] a) Segmentation

[0025] On the plane of the gray value, the information of the image liesessentially in the transition area between the black background and thewhite-represented retro-reflected mark. It should thus be consideredthat the image should be segmented into information-carrying parts andunimportant parts. This occurs with this image type in the simplest caseby a threshold formation with a subsequent dilation-operation, whichenlarges the light areas with a predetermined edge breadth R. Thereby itis ensured that the entire edge of the image point is detected.

[0026]FIG. 1 shows in detail a) a typical measurement image of asituation with retro-reflected marks, recorded with a Dalsa CA-D6high-speed camera. Besides the relatively large number of approximately170 marks with a low camera resolution of 512×512 image points theproportion of the information carrying image points represents only 3 to4%.

[0027] In part b) of FIG. 1 a histogram is shown of the occurringmarking frequency per image cell as an average of 70 images similar tothe scene in FIG. 1a). Maximally 22 marks per line were found, whereinthe marks were newly counted for each line. Overall a total of maximally176 marks were found in the entire image, on average there were 1.9marks per line.

[0028] The surface area of the image which is covered by marks dependsupon the mark diameter D, the edge breadth R and the number of marksn_(p). According to the segmentation the data amount corresponds to thefollowing number of bits: $\begin{matrix}{V_{s} = {n_{b} \cdot n_{p} \cdot \pi \cdot \left( {\frac{D}{2} + R} \right)^{2}}} & (9)\end{matrix}$

[0029] Therein n_(b) represents gray value for the bit depth. Thecompression factor γ is produced as a quotient of the image size{overscore (u)}·v and V_(s): $\begin{matrix}{\gamma_{s} = {\frac{\overset{\_}{u} \cdot \overset{\_}{v} \cdot n_{b}}{V_{s}} = \frac{\overset{\_}{u} \cdot \overset{\_}{v}}{n_{p} \cdot \pi \cdot \left( {{D/2} + R} \right)^{2}}}} & (10)\end{matrix}$

[0030] For an image size of 512×512 image points, n_(p)=170 marks with amark diameter of D=9 image points as well as edge R of 2 image pointsthere resulted a compression factor of γ≈11.6. This is 30 times less,lies close to the above approximation, is already sufficient in the caseof a rapid camera to intermediately store the image stream for asubsequent image measurement.

[0031] b) Gray Value Compression

[0032] A further significant reduction in the data amount is produced bycompression of the remaining gray values. This is particularly usefulabove all in combination with the above-described segmentation. Ofcourse, this step is associated with losses. The extent to which thisreduction methodology can be employed thus depends upon the degree ofprecision to be achieved in the subsequent image measurement. Themaximal reduction results in the borderline case of binarization, thatis, pixel values are only 0 (=dark) or 1 (=light).

[0033] The codification of the gray value from original gray value g tothe compromised gray value g_(c) occurs according to the followingequation: $\begin{matrix}{g_{c} = \left\{ \begin{matrix}{0\quad} & {{\quad \quad}{f\overset{¨}{{u\quad r\quad g} < g_{\min \quad}}}} \\\left\lbrack {{\frac{g - g_{\min}}{g_{\max} - g_{\min}} \cdot \left( {2^{n_{b}} - 2} \right)} + 1} \right\rbrack & {\quad {f\overset{¨}{{{ur}\quad g_{\min}} \leq g \leq g_{\max \quad}}}} \\{{2^{n_{b}} - 1}\quad} & {\quad {f\overset{¨}{{{ur}\quad g} > g_{\max \quad}}}}\end{matrix}\quad \right.} & (11)\end{matrix}$

[0034] The rectangular brackets represent the rounding off to the nextsmaller whole number. The gray value domain to be coded is bordered byg_(min) and g_(mas). The compromise gray value g_(c) then precisely hasa code for the informationless area (namely the 0) and a code for theover-illuminated area, that is 2^(n) _(b)−1. The remaining 2^(n) _(b)−2codes are essentially available for the remaining image information. Itis particularly desirable to employ such a code when the light range ofthe images has no great significance.

[0035] The reduction factor which can be achieved by gray valuecompression is simply the relationship of the bit-depth n′_(b) of theoriginal data to the bit-depth of the compromised data: $\begin{matrix}{{\gamma\kappa} = \frac{n_{b}^{\prime}}{n_{b}}} & (12)\end{matrix}$

[0036] In the binarization of a 12-bit deep image one achievesessentially a reduction by a factor of 12, and this only with asignificant gray value loss of the image measurement.

[0037] The process, which represents a combination of the segmentationand the gray value compression, is described by OBERPAUL (“Dynamic3D-Photogrammetrics”, MessTec & Automation 8(1), 67, January 2000).

[0038] c) Image Measurement

[0039] The most efficient, however also most complex, reduction methodis the image measurement itself. The essential information in the imageis the parameter which characterizes the signal marks. For a robustdescription of the marks one requires more than just the middle pointcoordinates {overscore (u)} and v. Further useful dimensions are alsothe surface A and the circumference 1 of the marks, as well as themaximal gray value g_(max) and the expansion along the coordinate axisΔu and Δv. Of further interest would be the ellipse parameters, that is,the two ellipticity coefficients a and b. The amount of informationcorresponds approximately 66 bit per mark, as summarized in thefollowing Table 1: Parameter Bits {overscore (u)} coordinate 12coordinate 12 Δu breadth 5 Δv height 5 A surface area 12 l circumference8 g_(mas) max. gray value 12 Sum: 66

[0040] The information requirement per image thus lies at:

V _(m) =n _(p)·66   (13)

[0041] In the above-mentioned example of 170 marks this would be a totalof 11220 bit or 1403 byes. The data reduction factor correspondsanalogously to the above equation (10): $\begin{matrix}{\gamma_{m} = {\frac{\overset{\sim}{u} \cdot \overset{\sim}{\upsilon} \cdot n_{b}}{V_{m}} = \frac{\overset{\sim}{u} \cdot \overset{\sim}{\upsilon} \cdot n_{b}}{66 \cdot n_{p}}}} & (14)\end{matrix}$

[0042] In the example according to FIG. 1 one achieves a data reductionfactor γ_(m)≈187. The higher the resolution of the camera, the moreefficient the process.

[0043] For a camera with 1320×1024 image points and 12 bit gray valueresolution there results already a factor γ_(m)≈1450.

[0044] d) Run Length Summation

[0045] An intermediate between the complexity of a complete imageanalysis and the above described image compromization is theline-by-line image measurement using run length summation. The datareduction presented to the computer therein considers respectively onlyone image point in the sequence as provided by the camera. Therecognition of the domains extending beyond the lines occurs in thecomputer. In the following Table 2 the parameters are indicated, whichare determinative in this process for further transmission to thecomputer: Σ g(u, v) gray value sum 16 Σ δu · g(u, v) center point moment20 δu breadth 5 g_(max) maximal gray value 12 ΔU run length 12 Sum: 65

[0046] The sequence of the data reduction occurs here according to thefollowing framework or routine:

[0047] 1. As long as b(u,v)=0, place δU=δU+1

[0048] 2. Send δU to the computer

[0049] 3. As long as b(u,v)=1, calculate Σ g(u,v), Σ δ u·g(u,v), andheight δ u as 1

[0050] 4. Send all sums to the computer, place ΔU=0 and begin again with1.

[0051] Thereby a simple condition automation with only two conditions isimplemented. The corresponding finite automation is represented in FIG.2. The transition between the two conditions S0 and S1 is controlled bya value of the binary image b(u,v) on the actual image coordinates.

[0052] For each mark, which appears in an image line, 65 bit must betransmitted. The computer now only has the task of melding the sums ofthe marks of two sequentially lines with each other, when they haveoverlapping starting and ends points. For calculating the moments, onetakes advantage of the fact that the line coordinates v over a line areconstant. By the use of relative coordinate δ u the breadth of themultiplier to be employed for data reduction can be significantlyreduced and thereby many resources can be saved. If one calculates fromthe relative coordinates again back to the absolute values, then fromthese moments the characteristic parameters can be calculated.

[0053] An image recognition system for photogrammetric analysis of thecenter of gravity of signal points with data reduction is described inthe U.S. Pat. No. 5,072,249. Therein the signals delivered by an analogvideo camera are first digitalized and subsequently submitted to adigital compression program or circuitry. In this program a datareduction occurs essentially according to the principle of run lengthsummation as outlined above. A further evaluation is subsequentlycarried out by the computer CPU.

[0054] The present invention represents an advance beyond this imagerecognition system as the closest state of the art. It is concerned withthe task of providing an improved system which makes possible a higherprocessing speed taking advantage of a new, further data reduction.

[0055] This task is solved by a process with the characteristicsaccording to claim 1. The characteristic features of an appropriatedevice for carrying out this inventive process are set forth in claim11. Further details of the invention and advantages of variousembodiments can be seen from the features of the dependent claims. Theinventive process and the corresponding device will be described in thefollowing on the basis of the preferred embodiment. Therein there isshown in

[0056]FIG. 3 possible binary bit pattern with 2×2 pixel groups

[0057]FIG. 4 a schematic representation of the coarse and switchconditions during the data reduction

[0058]FIG. 5 image of the circular mark in a binarized digital imagesection or segment

[0059]FIG. 6 block diagram of a digital circuit arrangement for datareduction

[0060]FIG. 7 function sequence of the control automation in the circuitaccording to FIG. 6

[0061]FIG. 8 design of the division or splitting ALU (Arithmetic LogicUnit) in FIG. 6

[0062]FIG. 9 design of the line ALU in FIG. 6

[0063]FIG. 10 alternative block diagram for calculating and intermediatestorage of various output values.

[0064] The basic concept of the invention is the use of sequential imagelines for a correlation analysis for data reduction prior to the actualfurther processing in a computer. Therein, groups of adjacent pixelsextending outside the line are evaluated. These groups are predefined inlines and columns with an identical number and distribution ofconsidered or evaluated pixels, for example, defined as n×m matrix.

[0065] In a binary image (that is, pixel values only 1 or 0) onlyspecific constellations of the distribution of pixel values within agroup are possible. These pixel patterns are identified and serve ascontrol signal for the actual data reduction.

[0066] For the below-described preferred embodiment a simple 2×2 matrixis given as pixel group. For the correlation analysis according theretotwo sequential lines of the binary image must always be analyzed. Thisrequires the intermediate storage of {overscore (u)}+2 image points inthe data shift register.

[0067]FIG. 3 shows the possible conditions of the binarized 2×2four-neighborhood of pixels. The overall 16 various combinations arerepresented as binary numbers, which serve as input signal for anevaluation logic. Various pixel patterns thereby cause varyingoperations of the evaluation logic, which can be represented as thefinal automation.

[0068]FIG. 4 shows a functional principle of the final automations formark measurement. In the shown condition diagram, the illustratedcircles symbolize the conditions, the arrows represent transitions. Thenumber on the arrows refer to the pixel pattern according to FIG. 3,which respectively is determined or measured as a binary number in theinput and triggers the transition.

[0069] The correlation analysis occurs via a register bank, of which thefree addresses in a column are used. If the automation recognizes thestart of a new mark, it retrieves the uppermost address from the columnand begins with the summation of the moments in corresponding element ofthe register bank. At the same time it writes the number of the usedregister for each image column u, which is covered by the mark, in adata shift register from the length {overscore (u)}+2. If in the nextline it arrives again at the same mark, it can recall the address in theregister bank from the data shift register. Each transition triggers orcauses a specific action of the automation. In the rectangular boxes inFIG. 4 the operations are indicated, which are initiated by thetransition:

[0070] New Mark (x): Get the address of an empty register bank from thecolumn and write the number in the data shift register position x andbegin with summation.

[0071] Summation (x): Sum the moments in the register bank from the datashift register entry x.

[0072] S (x): Unite the moment in the register bank from the data shiftregister entry x with the actual bank.

[0073] Finish (x): Write the moment from the register, which is noted inthe data shift register position x, in the output buffer.

[0074] The parameter x references respectively one double indirectaccess to a register bank. With x one of the four adjacent data shiftregister positions is indicated and from there the address which isnecessary for the operation is read in the register bank.

[0075] The automation has a total of 5 conditions S0. . . S4. In theline-by-line passage through of the image the automation recognizes onthe base of the bit pattern in the observed neighborhood of four 4,whether this concerns a new mark (transition 1) or whether a mark wascompletely detected (transition 6). For overview of the manner offunctioning, in FIG. 5 the enlarged image (20×20 pixel) of a circularshaped mark is represented, which in the following overview is sampledline-by-line, wherein the change in condition beginning with S0 is givenin the automation: CHANGE IN LINE COLUMN PATTERN OPERATION CONDITION 1/21/2 . . . 7/8 0 / S0 8/9 1 new mark (0) S0 −> S1  9/10 . . . 11/12 3summing (1) S1 12/13 2 / S1 −> S0 13/14 . . . 19/20 0 / S0 2/3 1/2 . . .5/6 0 / S0 6/7 1 new mark (0) S0 −> S1 7/8 3 summing (1) S1 8/9 7 sum(1) + S (2) S1 −> S2  9/10 . . . 11/12 15 summing (1) S2 12/13 11summing (1) S2 −> S1 13/14 3 summing (1) S1 14/15 2 / S1 −> S0 15/16 . .. 19/20 0 / S0 3/4 same sequence of pattern, operation and condition asline 2/3 4/5 1/2 . . . 2/3 0 / S0 3/4 1 new mark (0) S0 −> S1 4/5 7 sum(1) + S (2) S1 −> S2  5/6 . . . 15/16 15 summing (1) S2 16/17 11 summing(1) S2 −> S1 17/18 2 / S1 −> S0 19/20 0 / S0 5/6 1/2 . . . 2/3 0 / S03/4 5 summing (2) S0 −> S2  4/5 . . . 16/17 15 summing (1) S2 17/18 10 /S2 −> S0 18/19 . . . 19/20 0 / S0 6/7 same sequence of pattern,operation and condition as line 4/5 7/8 same sequence of pattern,operation and condition as line 5/6 8/9 1/2 1 new mark (0) S0 −> S1 2/37 Sum (1) + S S1 −> S2 (2)  3/4 . . . 17/18 15 summing (1) S2 18/19 10 /S2 −> S0 19/20 0 / S0 9/10  /12 5 summing (2) S0 −> S2  3/4 . . . 17/1815 summing (1) S2 18/19 11 summing (1) S2 −> S1 19/20 2 / S0 −> S1 10/11same sequence of pattern, operation and condition as line 5/6 11/12 samesequence of pattern, operation and condition as line 5/6 12/13 1/2 4 /S0 −> S4 2/3 13 summing (3) S4 −> S2  3/4 . . . 17/18 15 summing (1) S218/19 14 / S2 −> S3 19/20 8 / S3 −> S0 13/14 same sequence of pattern,operation and condition as line 5/6 14/15 same sequence of pattern,operation and condition as line 12/13 15/16 same sequence of pattern,operation and condition as line 5/6 16/17 same sequence of pattern,operation and condition as line 12/13 . . . / . . .

[0076] Continuation: CHANGE IN LINE COLUMN PATTERN OPERATION CONDITION17/18 1/2 . . . 3/4 0 / S0 4/5 4 / S0 −> S4 5/6 12 / S4 6/7 13 summing(3) S4 −> S2  7/8 . . . 13/14 15 summing (1) S2 14/15 14 / S2 −> S315/16 12 / S3 16/17 8 / S3 −> S0 17/18 . . . 19/20 0 / S0 19/20 1/2 . .. 8/9 0 / S0 9/10 4 / S0 −> S4 10/11 . . . 11/12 12 / S4 12/13 8Finished (3) S4 −> S0 13/14 . . . 19/20 0 / S0

[0077] 18/19 same sequence of pattern, operation and condition as line17/18

[0078] In this example of a function run or sequence no bit pattern oftype “6” or “9” according to FIG. 4 occurred, since only a single,complete bordered mark is the sample, in which this constellation is notpresent (see FIG. 5). Pixel patterns of this type could result from apartial overlapping of two signal marks and lead to correspondingoperations and transitions of the final automation (for example a pixelpattern “6” triggers the transition from S1 to S3 with operationsum(1)+S(2); pixel pattern “9” triggers either the transition from S3 toS1 or the transition from S4 to S4 with respective operation summing(3)). Therewith the final automation reliably also detects suchtransitions of two signal marks adjacent to each other in the image.

[0079] The introduced evaluation process for interline correlationanalysis by means of final automation can be realized by anappropriately designed electronic circuit. In the following the mannerof functioning of such an electronic circuit for measuring the center ofgravity of optical signal marks will be described on the basis of thepreferred embodiment according to FIG. 6.

[0080] The circuit works with digital input signals, for exampledirectly from a digital camera, and provides the results of theevaluation at the output likewise in digital form. The circuit musttherein solve multiple partial tasks: it must recognize the signal marksin the image, detect associated regions (segmentation) and calculate theassociated moments, from which the image measurement can be deduced. Forthis it is subdivided into multiple function blocks, as shown in FIG. 6.

[0081] At the input of the circuit signals arrive, which are producedeither by the camera or an external circuit. They are set forth in thefollowing Table 3: Short name Signal Bits Description SoF Start of Frame1 Image start SoL Start of Line 1 Line start PCLK Pixel Clock 1 Clocksignal for image point Data Pixel Data 8-12 Digital gray value

[0082] The bit breadth of the digital gray values, which are fed in viathe data lines, depend upon the camera which is to be used. Typicallythey are 8 or 12 bit. The Start-of-Frame (SoF) signal is used only forinitialization of the circuit at the beginning of each image and is thusnot shown in FIG. 6.

[0083] At the start of the circuit there is available informationregarding the individual signal marks which were contained in the image.This information is an image measurement hypotheses, from which in asimple processing step, for example, from a subsequent software or afurther electronic circuit, the actual image measurements can bededuced.

[0084] The moments determined by the circuit are indicated in Table 4:Name Description Σ g sum of the gray value Σ g · u moment of theu-coordinate Σ g · v moment of the v-coordinate Σ l surface area Σ l (u· v) circumference Δu breadth of the mark Δv height of the mark

[0085] From these moments then the image coordinates can be determinedby conventional calculation processes. These coordinates are the actualimage measurements in the photogrammetric sense. The other parametersare used only for recognition of reliable measurement. For example, therelationship of the circumference to surface area is a measurement forthe roundness of the mark. Longer objects can thus easily be sorted outas erroneous measurements.

[0086] These subsequent actions also can be carried out with the aid ofan electronic circuit or program (for example ASIC's). However thedevelopment expense therefore is relatively high and the need is low,since by this latter processing step the amount of data is hardlyreduced.

[0087] The function blocks shown in FIG. 6 have the tasks describedbelow:

[0088] There are certain global elements of which the signals can bemade available for the other blocks. These are on the one hand the linesand column counters u-counter and v-counter, which produce the line andcolumn numbers of the image from PClk, Start-of-Line and Start-of-Frame.Further, there is the B/W-converter, which compares the input gray valuewith a predetermined threshold value and provides at the output a binaryversion of the image. From the binarized signal there is intermediatelystored precisely one image line plus two image points in a data shiftregister (shift register). From this data shift register the finalautomation (control automation) extracts its most important inputsignal. Alternatively to the binarization with a threshold value onecould in this place also use the first derivative of the gray valuealong a line for this decision. If the difference of two adjacent grayvalues g₁-g₂ is smaller than one negative threshold value −t then thisconcerns an ascending flank in the image and all following points arebinarized as one. If the difference g₁-g₂ is greater than one positivethreshold value t, then this is a descending flank in the image and thesubsequent gray value is binarized as 0. This form of binarization ismore robust in comparison to the optical image caused light intensityoscillation of the image (for example vignetting or shading).

[0089] The control automation (finite state machine) has a fixedpredetermined number of 5 conditions S0. . . S4, which depend both onthe input signal from the data shift register as well as also from theprevious condition. For each condition a characteristic pattern ofoutput signals is produced (control) which serve for the control of theindividual circuit elements.

[0090] The automation is thus the “brain” of the circuit. Essentially itanalyzes as described above the condition of four direct adjacentbinarized image points from two sequential image lines. These four imagepoints can represent the 16 different combinations, as shown in FIG. 3.From the analysis of this pixel pattern the actions are deduced orinferred, which the other circuit blocks are carry out. Thus for examplea white point in the right lower element of a neighborhood of four(=pixel pattern 1) represents the start of a new mark.

[0091] The structure of the function sequence of the final automation ascontrol unit in this circuit design is shown in FIG. 7. In the showncircuit example, from the final (control automation) two arithmeticlogic units (ALU) are controlled, which can carry out simple arithmeticoperations (addition, subtraction, multiplication, exclusive-or). Thedesign of the column-ALU is shown in FIG. 8. The column-ALU calculatesand stores the moments for each mark in an image line and provides theresult to the line-ALU. For this it sums the moments for sequentiallight image points in one line. If in the binary image a transition fromlight to dark occurs, then all registers in the line-ALU aretransmitted.

[0092] The design of the line-ALU is shown in FIG. 9. The shown DATA INregister is identical with that in FIG. 8. The line-ALU combines thesums from two different sequential lines. For this it requires aring-buffer (dual ported RAM), in which all candidates from the previousline are intermediately stored. The values calculated from thecolumn-ALU are compared with those which are intermediately stored inthe ring-buffer and added if they originate from image areas whichconnect. This decision is made by the final automation on the basis ofthe observed neighborhood. The result is either written again in thering buffer for the next line or, in the case that the automationrecognizes that a mark has been completely detected, is transmitted tothe output buffer.

[0093] The output buffer assumes the approximately 142 bit wideinformation stream from the line-ALU and serializes it, so that it canbe transmitted to the output storage with the correct word breadth. Fromthis storage a connected computer can take over the data and carry outthe subsequent calculation for the image measurement.

[0094] The introduced or upstream data reduction by means of finalautomation can be realized in various ways as electronic circuit, ishowever not limited to the embodiment shown and described on the basisof FIG. 6. Accordingly FIG. 10 shows a somewhat modified circuitdiagram, which likewise achieves the described data reduction by meansof a final automat.

[0095] In similar manner the data reduction can be implemented usingField Programmable Gate Arrays (FPGA). In the case of FPGAs thisconcerns variable employable logical circuit loops, which areconstructed of a plurality of corresponding logic blocks (CLBs). Theyhave access to free programmable input and output, whereby they areemployable in multiple applications independently of pin setting andtransmission protocols. Typically employable are network routers andmobile radiotelephones. The programming occurs in built-in conditionduring the running operation and can be changed as often as desired upto 100 times per second. For frequent tasks, such as for example theconnecting to a PCI-Bus, finished program blocks are availablecommercially, which relieves the developer of a large part of theimplementation workload. The programming of the building blocks usuallyoccurs in a hardware description language such as HDL(Hardware-Definition-Language), VHDL (Very Large Scale IntegratedCircuit-HDL, for example VLSI-HDL) or AHDL (Applications SpecificIntegrated Circuit HDL, for example ASIC-HDL).

[0096] In the development of such a system it is to be taken intoconsideration that the expense for the development of a VHDL-Programmingwith lower flexibility is substantially higher than with a softwaresolution. It is thus important to select the intersection or interfacebetween hardware and software components of the solution, so that theend product runs sufficiently quickly with minimal development costs.

[0097] The above described data reduction with the center of gravitydetermination of optical signal markers by means of interlinecorrelation analysis by pattern dependent condition changes of anendless automation is very robust against disturbance or interferenceand can even measure non-convex areas. In general only a few parametersare required, so that the processing speed is correspondingly high.Because of the serial manner of operating also a parallelization bysimultaneous evaluation of different collated partial areas of an imageare possible. While this may necessitate an exception handling on theborders of the partial areas, nevertheless the processing speed cantherewith be increased, using digital electronics available today, toapproximately 500 million image points per second. Therewith thistechnique is also employable in the high-speed area.

1. Process for photogrammetrick determination of the centers of gravityof optical signal markers in moving objects, wherein the objects arerecorded by a video camera, of which the electronic image signal issupplied as line by line sequence of digitalized image points (pixels)to a digital processing unit for data reduction, of which the outputsignals are transmitted to a computer for further processing, therebycharacterized, that for data reduction groups of adjacent pixels ofsequential image cells are sequentially collated as n×m-matrix andevaluated.
 2. Process according to claim 1, thereby characterized, thatthe groups are combined as a 2×2 matrix.
 3. Process according to claim 1or 2, thereby characterized, that each group of adjacent pixels isevaluated as to whether the characteristic pattern of the distributionof the pixel values as lines and columns of the matrix are identified.4. Process according to claim 3, thereby characterized, that eachidentified pattern triggers an individual control signal.
 5. Processaccording to claim 4, thereby characterized, that the control signal issupplied as the input signal of a evaluation logic, which is designed asfinal automationwith a finite number of conditions.
 6. Process accordingto claim 5, thereby characterized, that the final automation carries outcondition changes depending upon the actual condition and input signal.7. Process according to claim 6, thereby characterized, that the finalautomation controls all operations of the digital processing unit fordata reduction.
 8. Process according to one of the preceding claims,thereby characterized, that all output signals of the digital processingunit for each signal mark the sum of all pixel values belonging to thesignal mark, indications of the position of the signal mark in theimage, surface area of the signal mark, the circumference of the signalmark, the breadth of the signal mark and the height of the signal markas digital numeric values for further processing in a computer arefurther conveyed.
 9. Process according to one of the preceding claims,thereby characterized, that the image signal supplied by the videocamera prior to the data reduction in the digital processing unit isreduced to a limited number of gray values.
 10. Process according toclaim 9, thereby characterized, that in one image cell the differentialvalue of the gray value of adjacent pixels are compared with apredetermined threshold value and in the case of a differential valuesmaller than the negative threshold value the gray of subsequent pixelsare set with a digital value “1”, in the case of differential valuegreater than the positive threshold value are determined as digitalvalue “0”.
 11. Device for photogrammetric determination of the thresholdvalue of optical signal marks of moving objects by means of the processaccording to one of the preceding claims, wherein the digital processingunit includes means for intermediate storage of at least one image cell,as well as a control unit, in which sequential image cell groups ofadjacent pixels are assembled or combined as n×m-matrix sequentially andevaluated as control signals, as well as means, which are controlled bythe control unit and further transmit the output signals to a computer.12. Device according to claim 11, thereby characterized, that a furtherincludes a conversion step or stage, in which the image signal suppliedby the video camera is reduced to a limited number of gray values. 13.Device according to claim 11 or 12, thereby characterized, that themeans controlled by the control unit represents arithmetic logic units.14. Device according to claim 11, 12 or 13, thereby characterized, thatfor components of the digital processing unit a field programmable gatearray (FPGA) or ASIC is employed.